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PERFORMANCE AUDITS 


Performance audits conducted by the Legislative Audit Division 
are designed to assess state government operations. From the 
audit work, a determination is made as to whether agencies and 
programs are accomplishing their purposes, and whether they 
can do so with greater efficiency and economy. 


We conducted this performance audit in accordance with 
generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for 
our findings and conclusions based on our audit objectives. We 
believe that the evidence obtained provides a reasonable basis for 
our findings and conclusions based on our audit objectives. 


Members of the performance audit staff hold degrees in 
disciplines appropriate to the audit process. Areas of expertise 
include business and public administration, journalism, 
accounting, economics, sociology, finance, political science, 
english, anthropology, computer science, international relations/ 
security, and chemistry. 


Performance audits are performed at the request of the Legislative 
Audit Committee which is a bicameral and bipartisan standing 
committee of the Montana Legislature. The committee consists 
of six members of the Senate and six members of the House of 
Representatives. 


Direct comments or inquiries to: 
Legislative Audit Division 
Room 160, State Capitol 
P.O. Box 201705 
Helena, MT 59620-1705 
(4.06) 444-3122 
Reports can be found in electronic format at: 


http://leg.mt.gov/audit 


Tori Hunthausen, Legislative Auditor 
Monica Huyg, Legal Counsel 


Deputy Legislative Auditors 
James Gillett 
Angie Grove 





September 2010 


‘The Legislative Audit Committee 
of the Montana State Legislature: 


This is our performance audit of the property reappraisal process managed by the 


Department of Revenue. 


This report provides the Legislature information about the property reappraisal 
process and includes recommendations to improve controls over reappraisal activities. 
Recommendations relate to strengthening documentation of property appraisals, 
defining expectations for the minimum number of properties to be inspected, improving 
documentation of supervisory reviews, and improving controls over the department's 


computer system to ensure accuracy and integrity of data related to property valuation. 


We wish to express our appreciation to the department director and staff for their 


cooperation and assistance during the audit. 


Respectfully submitted, 
// Tori Hunthausen 


Tori Hunthausen, CPA 
Legislative Auditor 


Room 160 ¢ State Capitol Building * PO Box 201705 * Helena, MT * 59620-1705 
Phone (406) 444-3122 * FAX (406) 444-9784 * E-Mail lad@mt.gov 
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Reappraisal Process 
Department of Revenue 


10P-11 REPORT SUMMARY 


Bolstering property appraisal documentation, regularly inspecting the state’s 950,000 


properties and protecting the integrity of appraisal data will enhance the Department of 
Revenue’s property reappraisal process — a process which helps generate more than 
$200 million annually for the state’s general fund. 








Context 


Montana is only one of two states that are a “mass 
appraisal” state. Mass appraisal is the process of 
valuing a group of properties as of a given date using 
common data and standardized methods. A mass 
appraisal system gathers data using various methods 
such as physical inspection of properties, computer 
models, and aerial photography. The Department of 
Revenue is responsible for reappraising all real 
property for tax purposes on a six-year cycle. Because 
of the complexity of the department’s reappraisal 
process, it is governed by a number of state laws, 
administrative rules, department policies and 
procedures, and professional appraisal standards. 


The department’s property reappraisal responsibilities 
are performed by staff located in Helena’s Property 
Assessment Division and four geographic regions 
around the state. Every county in the state has a local 
department office which is located in the county seat. 
The department is required to appraise residential 
properties, including the land and any improvements. 
Appraisers review and document the physical 
characteristics, condition, desirability, and utility of 
the property. Ideally, an internal or external physical 
inspection of the property is desired. The value of the 
property is calculated using either a sales approach or 
a cost approach. The department is currently 
responsible for reappraising more than 950,000 
parcels of property statewide. 


Property Reappraisal Dataflow 





Source: Compiled by the Legislative Audit Division from 
Information obtained from the Devartment of Revenue. 





In the event a property owner is dissatisfied with the 
appraised value or classification of their land or 
improvements, state law allows for taxpayers to appeal 
the department’s appraisal. Appeals can be made to the 
department, to the County Tax Appeal Board, or State 
Tax Appeal Board. 


We had two objectives for this audit. One was to 
determine if the property reappraisal process was 
consistently followed statewide. The other was to 
determine if controls existed to ensure the integrity 
and accuracy of reappraisal data. Audit work included 
an evaluation of the department’s procedures for 
appraising and documenting property values. We also 
examined the department’s methods to conduct 
department reviews when property owner’s dispute the 
department’s appraised value of their property. The 
controls over the department’s computer system 
(Orion) which maintains and calculates property value 
information were analyzed to assess the accuracy of 
data maintained by the system. Audit work generally 
focused on the six-year property reappraisal cycle 
beginning January | 2002, and ending July 1, 2008, 
(these reappraisal values were used for property tax 
purposes in tax year 2009). To meet our objectives, we 


examined documentation for a random sample of 200 
property appraisal files for residential, commercial, 
agricultural, and forestland properties located in five 
counties. Counties visited included Flathead, Gallatin, 
Lewis and Clark, Treasure, and Yellowstone. We 
queried Orion data and compared this data to property 
appraisal documentation maintained by local 
department offices. We also interviewed department 
management and staff located in local offices and in 
Helena. We used provisions identified in statute, 
administrative rule, department policies and 
procedures, and professional appraisal standards as 
criteria in which to evaluate the department’s 
reappraisal process. 


Results 


Audit work noted appraisers follow similar processes 
to appraise property, but controls over the process 
should be improved. It was not always possible to 
verify the accuracy of information used to calculate 
property values due to inconsistencies in file 
documentation maintained in the department’s local 
property record files. For example, information related 
to property characteristics (square footage, condition 
and utility of property, etc.) or why the department 
used a property value override was not always 
documented. We also noted approximately 26 percent 
of properties were not physically inspected during the 
last six-year reappraisal cycle with property 
inspections ranging from 19 percent to 99 percent. 
Orion (the department’s computer system) maintains 
and calculates property values for the department. 
However, audit work noted several control weakness 
related to the system which could result in processing 
or valuation errors related to property values. 


To address these concerns, our audit made several 

recommendations to improve management of the 

property reappraisal process. Recommendations issued 

are related to: 

> Better documenting property appraisals 

> Defining expectations for the number of property 
inspections conducted during each reappraisal 
cycle 

> Complying with department review procedures of 
disputed appraisals 

> Standardizing documentation of supervisory 
reviews of appraisals 

> Limiting uncontrolled employee access to Orion 
data used to calculate property values 

> Developing procedures to ensure department staff 
recelve management authorization to override 
property values maintained in Orion 

> Establishing controls to ensure property value 
data entered in Orion is complete and accurate 


Recommendation Concurrence 








Concur 9 
Partially Concur 0 
Do Not Concur 0 














Source: Agency audit response included in final report. 





For a complete copy of the report or for further information, contact the 
Legislative Audit Division at 406-444-3122; e-mail to lad@mt.gov; or check the website at http://leg.mt.gov/audit. 





Report Fraud, Waste, and Abuse to the Legislative Auditor’s FRAUD HOTLINE 
Call toll-free 1-800-222-4446, or e-mail lad@mt.gov. 





Chapter | — Introduction 


Introduction 


The Montana Constitution and state law require periodic reappraisal of property in 
the interest of equal taxation. The State of Montana, through the Department of 
Revenue, is responsible for valuing all taxable real and personal property. According to 
§15-7-131, MCA, it is the policy of the state to provide for equitable assessment of 
taxable property in the state and to provide for periodic revaluation of taxable 
property in a manner that is fair to all taxpayers. Section 15-7-111, MCA, requires 
the department to conduct a cyclical revaluation of residential, commercial, industrial, 
agricultural and forestland property in the state. The Legislative Audit Committee 
requested a performance audit of the department’s process to periodically reappraise 


property. 


Audit Objectives 


Based on our initial assessment of the property reappraisal process, we developed two 
audit objectives. To determine: 


1. If the department’s property reappraisal process is consistently followed 
statewide. 


2. Ifcontrols exist to ensure integrity and accuracy of reappraisal data. 


Audit Scope 


The scope of this audit focused on the department's reappraisal process for four property 
classes: residential, commercial, agricultural and forestland property. Audit work 
included an evaluation of the department's procedures for assessing and documenting 
property values. Our analysis included an examination of the department’s methods 
for processing requests by property owners for department reviews to dispute appraised 
property values. We also analyzed controls over the department computer system 
(Orion) which maintains and calculates property valuation information. Controls 
tested included those related to data input, access to the system, and accuracy of 
property appraisal data. The scope of our audit generally focused on the six-year 
property reappraisal cycle beginning January 1, 2002, and ending July 1, 2008. These 
reappraisal values were used for property tax purposes in tax year 2009. 


Our audit did not include a review of industrial or personal property classes. We 
also did not evaluate the accuracy of individual appraisals or review the property tax 
mitigation strategy implemented by the 2009 Legislature to help lessen the impact of 
the most recent property reappraisal. Audit scope also excluded the processes related to 
property tax billing, collection, and reconciliation. 
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Audit Methodologies 


To address the audit objectives we conducted the following work: 


a 


Reviewed applicable statutes, administrative rules, and department policies 
and procedures to identify guidance available to department staff. 


Reviewed appraisal standards from the International Association of Assessing 
Officers (IAAO), which are the professional standards followed by the 


department when conducting appraisals. 
Reviewed current certifications of appraisal staff. 


Examined documentation for a random sample of 200 property appraisal 
files for residential (67 files), commercial (53 files), agricultural (44 files), and 
forestland (36 files) properties located in five Montana counties. Counties 
visited included Flathead, Gallatin, Lewis and Clark, Treasure, and 
Yellowstone. 


Queried Orion data and compared data to property appraisal documentation 
maintained by local department offices. 


Reviewed system documentation for Orion related to processing, data input, 
system access, and the integrity of property data. 


Reviewed documentation related to informal review requests filed for the 
2009 tax year in the five counties identified above. 


Observed local department staff conduct informal reviews with property 
owners. 


Attended and observed County Tax Appeal Board and State Tax Appeal 
Board hearings. 


Interviewed department staff and management located in both local offices 
and in Helena. 


Contacted other states to obtain information regarding the property appraisal 
processes in those states. 


Management Memorandum 


A management memorandum is a verbal or written notification to the agency of issues 


that should be considered by management, but do not require a formal agency response. 


We discussed an issue with the department regarding steps needed to further improve 


its appraiser certification process. We also discussed the need for the department to 


consistently follow its established controls to verify department staff are not appraising 


their own property or property of immediate family members. 


Report Contents 


The remainder of this report includes a background chapter followed by chapters 


detailing our findings, conclusions, and recommendations in the following areas: 


Chapter II presents recommendations for improving controls over the 
property reappraisal process. 


Chapter IV discusses needed improvement of controls over the integrity of 
property data maintained in the department's property reappraisal computer 
system. 


Chapter II - Background 


Introduction 


In 1975, the Legislature enacted statute requiring the Department of Revenue to 
administer and supervise a program for the revaluation of all real taxable property 
in Montana. ‘The legislature first established cyclical reappraisal of real property on 
a five-year schedule. This has since been changed to a six-year cycle and there have 
been six reappraisals completed in Montana since 1975. Reappraisal cycles are 
designed to ensure all property is taxed on current structural and market information. 
The department's role in the cyclical reappraisal process is to supervise the property 
reappraisal process, conduct property appraisal, oversee the input of property 
characteristics into Orion, and ensure all classes of property in the state are valued 
uniformly. This chapter describes the department’s organizational structure and its 
property valuation responsibilities. 


Reappraisal of Property 


Montana is one of only two states that are a “mass appraisal” state. Mass appraisal is 
the process of valuing a group of properties as of a given date by using common data, 
standardized methods, and statistical testing. A mass appraisal system gathers data 
using various methods such as physical inspection of individual properties, computer 
models, aerial photography, etc. Because of the complexity of the reappraisal process, 
it is governed by a number of state laws, administrative rules, professional appraisal 
standards, and department policies and procedures. According to department 
information, the number of parcels of property in Montana increased almost 9 percent 
over the last six years, increasing from approximately 875,000 parcels in 2003 to more 
than 950,000 parcels in 2009. The following sections provide information regarding 
the department's property reappraisal responsibilities and describe its process to 
appraise residential, commercial, agricultural, and forestland properties. The following 
sections also discuss the appeals process to dispute appraised property values and the 
department’s computer system used to administer the property reappraisal process. 


Department Property Reappraisal Responsibilities 


The department is responsible for administering Montana’s property tax laws, 
including the valuation and assessment of all real and personal property in Montana for 
property tax purposes. According to the department's biennial report, the department 
is responsible for ensuring all classes of property (residential, commercial, agricultural, 
etc.) are valued uniformly and equally throughout the state. Department responsibilities 
are performed by staff located in Helena’s Property Assessment Division and four 
geographic regions around the state. The following figure illustrates the department's 
property appraisal organizational structure. 
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Figure 1 
Property Appraisal Organizational Structure 





Deputy Division Management 
Administrator Officer 


Region 1 Region 2 Region 3 Region 4 Eade rt 
Manager Manager Manager Manager Sea 
Area Area Area Area 
Manager Manager Manager Manager Management 
Analysts 
Appraisal Appraisal Appraisal Appraisal 
Staff Staff Staff Staff 


Source: Compiled by the Legislative Audit Division from department records. 





Regional Managers are responsible for administering operations in each region. In 
June 2010, the department consolidated the number of regions from six to four. The 
department is authorized approximately 315 FTE for property appraisal duties. Most 
of these positions are located in local field offices in the four regions around the state. 
The central office in Helena has a total of 20 FTE with the remaining assigned to local 
offices. The following figure illustrates the department's geographic regions related to 
its property reappraisal process and counties visited during the audit. 
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Every county in the state has a local department office which is located in the county 
seat. County offices are grouped together geographically to form areas. For example, 
Flathead and Lake Counties are considered an area within Region One in northwest 
Montana. Area Managers provide day-to-day oversight of reappraisal activities within 
their assigned area. 


Residential Property Appraisals 


The department is required to appraise residential properties, including the land and 
any improvements. As part of the review, appraisers consider the physical characteristics 
of the property (¢.g., square footage, materials, additions, etc.) as well as the condition, 
desirability, and utility of the property. Ideally, a physical internal inspection of the 
property is desired; however, an external inspection will suffice. Property characteristics 
are documented by the appraiser that visited the property. 


Using the updated information, a new property value is calculated using one of two 
approaches. For the sales comparison approach, models are used to compare specific 
property to other similar properties in a given geographic area sold within a specific 
timeframe. The property’s value is adjusted based on the sales prices of the other 
properties and any differences between the properties. This approach is the preferred 
method for determining value, but may not be applicable in all situations, in which 
case the cost approach is used. The cost approach values a property based on the 
current replacement cost of the improvements, minus depreciation, plus land value. 
After calculating the property’s total value using both valuation methods, an appraiser 
decides the final value of the property by determining which valuation approach is 
most appropriate. Appraisers use their judgment, training, and department resources 
to conclude which approach will be used. The department contracted with a private 
firm to conduct a market price trend and sales ratio study of the department’s 2009 
residential property reappraisal. The study indicated the department met an [AAO 
standard of having a sample appraisal level within 10 percent of market value. The 
report also indicated appraisal uniformity was generally good. 


Commercial Property Appraisals 


When conducting commercial property appraisals, the appraiser conducts a field 
inspection (an internal or external observation of the property) and records the 
property’s characteristics. Characteristics of commercial property include, but are 
not limited to: structure type, year built, number of units per building (e.g. storage 
units or apartment complexes will have multiple units per building), square footage of 
building, etc. Using this information the value of the property is calculated. 


When determining the value of a property, commercial appraisers generally use one 
of two methods-cost approach or income approach. When determining the property’s 
value using the cost approach, the value is based on a cost approach model developed 
by the department for commercial property. The cost approach model is developed 
using cost tables which provide estimates of the cost of building similar improvements. 
In general, the cost approach uses the current replacement cost of improvements, less 
depreciation, plus land value. 


The income approach is used to value commercial properties when income and 
expense information is available. To obtain this information, the department sends 
income-expense report forms to all commercial property owners. When property 
owners return these forms, the information is compiled into models used to value 
commercial properties. Similar to residential properties, the appraiser determines 
which property value, that calculated using the cost approach or the income approach, 
is most appropriate. It should be noted the information returned by property owners 
may not necessarily be used to value their property. The department prefers to value 
commercial property using the income approach when possible. However, according 
to department staff, the return rate of income-expense report forms is approximately 
15 percent. The department’s contracted study indicated commercial properties for the 
2009 reappraisal cycle were also close to market value. 


Agricultural Land Property Appraisals 


Until this year, the method used for determining agricultural land values had not 
changed since the early 1960s. When the department assumed statewide responsibility 
for determining land values, the department assimilated existing county land-use 
determinations into its process. In the intervening years, those county-determined 
valuations were only changed to reflect the increased value of the commodities being 
produced by the land. However, with this reappraisal cycle, the department attempted 
to equalize agricultural land values across the state. The goal of this equalization was 
to ensure agricultural lands with the same productive capacity and used for the same 
purposes will be taxed the same, regardless of the county in which they are located. 


Determining agricultural land value is based on three variables: size of the parcel, the 
agricultural classification of the parcel, and the soil’s productive capacity. Parcels larger 
than 160 acres are automatically classified as agricultural, unless they were previously 
classified as another property type. Any parcel of land that is smaller than 160 acres 
must pass certain use and income tests before it can be classified as agricultural. Once 
it has been determined the land qualifies for agricultural valuation, its use is classified 
as one of five land-use classifications. Different use classifications result in different 
productive value. For example, lands that are tillable are valued higher than those used 
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for grazing. The last part of the process is to determine how productive the land is across 
the state. Through coordination between the department, several federal agencies, and 
the Governor’s Agricultural Land Valuation Advisory Committee, the department now 
relies on soil surveys from the USDA Natural Resource Conservation Service (NRCS). 
NRCS has developed a database of soil quality across the state. Depending on the land 
use classification, soil quality can identify how much of a given commodity can be 
produced. To further standardize the process, the department, through the Governor's 
Agricultural Land Valuation Advisory Committee, has established certain commodity 
crops as the default crops when determining land values, regardless of whether the 
landowner is producing that crop. 


Based on the 2009 reappraisal, the department has identified some methods for 
potentially improving its new agricultural reappraisal process. Staff indicated there are 
plans to consider topography and buildup of salinity due to irrigation practices and the 
resulting effects on productivity in future reappraisals. 


Forestland Property Appraisals 


In order for land to be classified as forestland, it must consist of at least 15 contiguous 
acres of land capable of producing a minimum of 25 cubic feet of timber per acre per 
year. In order to equalize the taxed value of all forestlands throughout the state, the 
department sought assistance from the University of Montana’s School of Forestry 
to determine productivity levels of the state’s forested lands. The School of Forestry 
developed a model based on 12 different variables (land slope, average solar coverage, 
rainfall, etc.) to determine how productive the forest soils are, and therefore how much 
timber could be produced on the property. According to the department, the work 
completed by the School of Forestry resulted in the development of soil productivity 
levels for each 30 square meters of forested property in the state. To determine the land 
value, the amount of timber that can be produced by the property per acre, based on 
its productivity level, is multiplied by the number of acres. 


The Appeals Process 


In the event a property owner is dissatisfied with the appraised value or classification 
of their land or improvements, §15-7-102, MCA, allows for the taxpayer to appeal 
the department’s appraisal. Appeals can be made to the department or formally to 
the County Tax Appeal Board (CTAB) or State Tax Appeal Board (STAB). Per 
state law, department staff conducts a department review of a property’s value and 
classification upon request by the property owner. The taxpayer must submit a request 
to the local field office within 30 days of the date of the property assessment notice. 
Upon receipt of the request by the department, the information is then assigned to an 
appraiser for review. In the case of an appeal of productivity levels for agricultural or 


forestland, the request is forwarded to the department’s central office for review. There 
were approximately 40,000 department review requests filed in response to the 2009 
reappraisal cycle. 


If the taxpayer is unsatisfied with the results of the department review, they may 
appeal to the local CTAB. Taxpayers may also appeal directly to the CTAB without 
first requesting a department review. If either party (the department or the taxpayer) 
is unsatisfied with the results of a CTAB hearing, an appeal can be made to STAB 
within 30 days of the CTAB decision. 


Orion Maintains and Calculates 
Property Value Information 


To assist in the administration of property reappraisal, the Department of Revenue 
purchased a commercial off-the-shelf property appraisal computer system called 
Orion. Orion maintains and calculates property value information for the department. 
The system was first implemented in 2007, one year before property valuation was 
completed. The system, originally designed to be used by counties and smaller 
municipalities to value properties, has been customized by the department to be used 
for statewide property reappraisal. Department field staff in each county is responsible 
for entering property characteristics into the system, which will determine the value 
of the property. The following figure illustrates the dataflow of property reappraisal 
information. 


Figure 3 
Property Reappraisal Dataflow 
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Source: Compiled by the Legislative Audit Division from information obtained from the Department of Revenue. 





Initially, Orion was managed by the department’s Property Assessment Division. 
Currently, management responsibilities are shared by the division and the department's 
Information Technology & Processing Division. The Property Assessment Division 
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handles the maintenance and change management of the system, while Information 
Technology & Processing is responsible for the security elements. 


Conclusion: Reappraisal Process Is Consistent But 
Management Controls Could Be Strengthened 


Our primary objective was to examine the consistency of the property reappraisal 
process across the state. We found appraisers in the counties we visited followed similar 
processes to appraise property. The department's process for appraising agricultural 
and forestland property without improvements, such as a home, was conducted in 
a consistent manner. However, audit work concluded that the department could 
strengthen management controls over the property reappraisal process for other 
property classes, such as for residential and commercial property. Examples of issues 
identified included appraisals that were either not documented or inconsistently 
documented, selection of properties for physical visitation during the reappraisal cycle, 
and methods to perform department reviews when property values are questioned. In 
addition, we identified limited controls over the integrity and accuracy of reappraisal 
data maintained in Orion. The following chapters discuss these issues in more detail 


and present recommendations to improve controls over the reappraisal process. 


Chapter III - Controls Over The 
Property Reappraisal Process 


Introduction 


Consistent and accurate valuation of property requires correct, complete, and 
up-to-date property information. Organizations responsible for valuing property must 
establish effective procedures for collecting and maintaining property information 
(e.g. location, size, physical characteristics, income, costs, etc.). Based on our audit 
work, we identified areas where the department can enhance its controls for appraising 
and valuing real property. Audit findings and recommendations relate to the following 


areas: 
¢ Strengthen documentation to support property appraisal 
¢ Standardize identification of property inspections 
¢ Processing department review requests 
¢ Document supervisory review of appraisal activities 


¢ Improve management information for future decision-making in the 
appraisal process 


Strengthening the Documentation 
Behind Property Appraisal 


IAAO standards require a property appraisal process be well documented to help 
ensure accountability and transparency of the process. The standards recommend an 
assessing organization collect property characteristics data in the field sufficient for 
classification, valuation, and other purposes and facilitate its entry into an information 
system. In addition, department policy outlines procedures for ensuring physical 
characteristics recorded in the field are retained and input into the department’s 
information system. As part of audit work, we examined the process used by department 
staff to document information regarding the valuation of property. We selected five 
counties and reviewed a sample of property record files for parcels in four property 
classes—residential, commercial, agricultural, and forestland. As part of our review, we 
compared information contained in the department’s computer system (Orion) to the 
physical documentation in the files related to the appraisal of the properties within our 
sample. We noted inconsistencies in various types of file documentation throughout 
the sample, primarily related to appraisals of residential and commercial property. 
Consequently, it was not always possible to verify the accuracy of the information in 
Orion, which was used to calculate the values of the sampled properties. ‘The following 


examples discuss documentation inconsistencies identified: 


CO SS 


13 


14 


| Montana Legislative Audit Division Legislative Audit Division 


¢ Property Characteristics — We noted approximately 30 percent (35 out 
of 120 files) of residential and commercial property records, located at 
department county offices, contained property characteristic information 
which did not match data contained in Orion. 


¢ Valuation Approach Selection — While we were able to determine which 
valuation approach was used for all sampled properties, none of the property 
record files or Orion contained documentation as to why the selected 
valuation approach was chosen to calculate the property’s final value. 


¢ Qualification as Agricultural Land — We found the department accepts 
various forms of proof of income from taxpayers to qualify land less than 
160 acres as agriculture land. In addition, the presence of documentation of 
income in property record files was inconsistent. 


¢ Walue Overrides in Orion — We identified over 2,400 instances where 
property value overrides were used and designated only as “other” or 
“miscellaneous” with no other supporting documentation provided. 


¢ Department Review Process — When property owners were dissatisfied 
with their appraised value and requested a department review, there was 
limited documentation indicating a property inspection occurred during the 
review process for nearly 13 percent of the closed department reviews. 


¢ Property Value Adjustments — There was limited documentation why 
property value adjustments were made during the department review process. 
In over half of the property record files associated with the department 
reviews selected for our sample, we found no such documentation in Orion. 


Conclusion: Overall Documentation Could Be Improved 


The inconsistency of documentation regarding the issues noted was observed in 
all counties we visited. Not documenting key actions or decisions employed in the 
valuation of property may potentially impact the department's ability to ensure its 
property assessment activities result in fair and equitable taxation across the state. ‘This 
decreases the transparency of the appraisal process for taxpayers and during appeal 
situations. The department should strengthen controls for documenting property 
information during the reappraisal process. 


Mi 


RECOMMENDATION #1 





We recommend the Department of Revenue strengthen controls for 
documenting reappraisal activities. 


TO 


Department Identification of Properties 
for Physical Inspection 


We found department staff generally conducts the physical inspection of properties in 
a similar manner; however, we noted variances across the state in how many properties 
are inspected in each county. We extracted data from Orion related to all properties 
and their most recent physical inspection. According to this data 26 percent of all 
properties in Montana were not physically inspected during the last six-year reappraisal 
cycle. In addition, the number of properties not physically inspected varied significantly 
between counties. For example, department data shows physical inspection of property 
ranged from 19 percent of properties in one county to 99 percent in another. We also 
noted 41 properties which had not been inspected during the last three reappraisal 
cycles. The following map illustrates the percentages of properties physically visited by 
the department for each county during the last reappraisal cycle. 
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Figure 4 
Properties Physically Inspected By Department Staff 
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Physical Inspection is an Important 
Aspect of Determining Value 


The department is not required by statute to physically inspect all properties during 
each reappraisal cycle. However, [AAO standards on mass appraisal emphasize the 
importance of timely physical inspection: 


“...An accurate valuation of real property by any method requires description 
of land and building characteristics. The primary way to obtain property 
characteristics is to physically inspect the property...properties should be 
physically reviewed and individually reappraised at least every four to six 
years... 


The majority of department staff interviewed indicated it is the department’s duty 
to physically inspect all property during each reappraisal cycle; however, most noted 
that it was not likely all properties in Montana were physically visited. A variety of 
reasons were offered by staff. It was indicated that, because of the centralized methods 
of valuing agricultural and forestland, those properties are not generally visited unless 
an improvement is located on the property. 


Not physically inspecting property increases the risk that property characteristics used 
to calculate a property’s final value are not complete and current. [AAO standards 
indicate a system should be developed for identifying properties for physical inspection 
to ensure recorded property characteristics are complete and accurate. Currently, the 
department has not clearly defined expectations for the minimum number of property 
inspections to be conducted. Expectations could be developed for different property 
classes or regions of the state. 


The department should balance the need for physical inspection of property and the 
resources available to determine how it can most effectively carry out its duties. The 
importance for determining the minimum number of properties to be inspected and 
where property inspections should occur is even more important when resources are 
limited. The department may determine that physical inspection of every property 
is not necessary during each reappraisal cycle; in that case, a strategy for equalizing 
the likelihood of property inspection should be implemented. If the department 
determines inspection of every property is essential, other methods of inspection, such 
as the use of contracted appraisers or data collectors, may need to be considered. 
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RECOMMENDATION #2 





We recommend the Department of Revenue define expectations for the 
minimum number of property inspections to be conducted during each 
reappraisal cycle. 


TO 


Processing Department Review Requests 


The department review process is an important part of the reappraisal of property 
as it allows taxpayers an opportunity to protest property values. As part of our audit 
work, we examined the timeliness with which the department issued decisions as a 
result of these reviews. We found the average time between the date the department 
review request was filed by the taxpayer and the issuance of a final decision by the 
department was approximately 128 days. Timeframes varied between residential and 
commercial property; however, these variances may be explained by the nature of the 
appraisal of each property type, as commercial property tends to present a higher level 
of complexity than residential property. The length of processing time for department 
reviews of agricultural properties was longer, with one case reviewed taking 335 days 
for resolution. The increased timeframe is in part due to the department’s decision to 
process all agricultural review requests related to land productivity in Helena. The 
lengthy time period for processing department review requests increases the likelihood 
that property owners may not be getting timely resolution of appeals. 


Department Procedures Should be Followed 


IAAO standards provide some guidance on the timeliness of property tax appeal 
processes and state that sufficient time should be provided for property owners to 
protest assessments and for assessing entities to render a decision. The standard also 
notes, however, the time period provided for appeals should not be so long as to delay 
tax collections unreasonably. In Montana, statute does not delineate a timeframe for 
the processing of department review requests. Department procedure requires the 
department to respond to all requests for a department review within 30 days. The 
procedure further states extensions to the thirty-day requirement may be granted 
if agreed upon by all parties; however, we found no documentation indicating the 
thirty-day response timelines or extension requirements were met. 


RECOMMENDATION #3 





We recommend the Department of Revenue comply with current procedure 
for department reviews. 


Document Supervisory Review 


IAAO standards recommends supervisory review of appraisal work. However, 
department policy does not refer to documenting supervisory review during the 
property reappraisal process. In our review of the property record files selected for 
our sample, we found limited existence of documentation substantiating supervisory 
review of appraised values. Within our sample of 200 property record files, less than 
five percent exhibited documentation of supervisory review of property reappraisal 
activities. In addition, for our sample of 74 department review files, we found the 
extent of documentation of supervisory review varied in the counties we visited. Due to 
the decentralized nature of the property reappraisal process, documenting supervisory 
oversight is needed to ensure accountability in the process. 


RECOMMENDATION #4 





We recommend the Department of Revenue establish standardized methods 
for documenting supervisory review. 


Management Information Needed for 
Future Decision-Making 


During the course of our audit, we noted the department has been limited in their 
capability to compile management information related to the property reappraisal 
process due to data system limitations. Examples of management information which 
could help the department improve management of the reappraisal process include: 


¢ — Identification of properties which were not visited. 


¢ Identification of properties affected by natural disasters in order to ensure 
appropriate adjustments to neighboring properties. 


¢ ‘The percentage of change for each property value adjusted and the overall 
percentage of change for all property values adjusted during department 
reviews and appeals. 


¢ Whether property values increased or decreased as the result of department 
reviews and appeals. 
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¢ ‘The reasons for adjustment of property values as the result of department 
reviews and appeals. 


¢ Identification of extreme property value adjustments as the result of an 
department review. 


The ability to access and analyze program information is a vital tool to be used by agency 
management for making key decisions regarding budgeting, staffing, and program 
direction. Although department staff noted difficulties extracting standardized reports 
from Orion, management relies on this information for reporting on agency and 
program activities to the Legislature, cooperative organizations, and the public. 


In addition, appraisal information can be used to identify program weaknesses. 
Failure to make use of this information increases the potential the department is 
missing opportunities to identify problem areas and improve the appraisal process 
based on management information. For example, lack of property visitation, failure 
to properly certify appraisers, or inconsistency of documentation practices during the 
initial appraisal of a property may contribute to property value adjustments during the 
informal review or appeal processes. Management information about the reasons for 
property value adjustments may help the department identify these types of trends. In 
addition, identification of properties not recently inspected aids managers in prioritizing 
those properties for inspection. Management information regarding property appraisal 
and the outcome and trends associated with department reviews and appeals should 
be used by the department to better manage the overall appraisal process and identify 
“problem areas” requiring attention. 
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RECOMMENDATION #5 


We recommend the Department of Revenue identify, compile, and analyze 
management information to improve the property reappraisal process. 


TO 


Chapter IV — Integrity of Orion 
Property Reappraisal Data 


Introduction 


Orion, the department’s computer system, plays a key role in the maintenance of 
property tax records and the calculation of property assessments. As the primary 
administrative tool in the property reappraisal process, the integrity of Orion 
processing and data is critical to accurate assessment values. It is imperative the system 
is accurately processing and calculating tax-related transactions and maintaining the 
integrity of property tax data. According to department staff, information in Orion 
is considered the official record for property values. Therefore strong system controls 
are required in the areas of change control, user access, data input, and data output. 
Overall, we found the Department of Revenue has implemented controls to maintain 
data integrity. For example, the department had an understanding of the system and 
ensured it was working as expected prior to implementation. In addition, in the area 
of data output, the department has implemented a new system (Oasis) to generate 
property assessment notices. Audit testing indicates this process ensures a complete 
transfer of Orion data to Oasis. However, we identified control weaknesses that present 
a risk to the accurate processing of property values. This chapter addresses our objective 
regarding the integrity and accuracy of reappraisal data. It discusses controls currently 
in place, as well as identified weaknesses, including inconsistent change management, 
uncontrolled system access, and limited data input control. 


Identified Control Weaknesses Impact Orion Processing 


If the delivered system does not meet the needs of the department, enhancements 
can be made to provide more customization. The department has made a number of 
modifications to Orion since implementation. These modifications are primarily the 
result of changing requirements for calculating values of the various property types 
and additional customization of Orion to better suit the needs of the agency. A number 
of modifications have been made to the system to address the added complexities of 
property assessment performed on a statewide basis as opposed to a county-wide basis. 
Information Technology (IT) standards indicate a change control process needs to be 
established, which includes request, development, testing, and approval of all changes 


prior to implementation into production. 


Inconsistent Change Control Process 


The department created a standard document to use when Orion requires a 
modification or change. After reviewing a sample of these documents, we identified 
areas of the change control process that were either not completed or performed 
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incorrectly according to IT standards. After reviewing ten random change tracking 
documents, we identified only one with management approval and zero that had 
signatures denoting agency management acceptance of the change. The vendor was 
responsible for the development of changes, but it was the responsibility of the agency 
to test new changes to ensure the change met expectations and did not negatively 
impact the system. 


Although all of the changes we reviewed were tested, only half were tested in a test 
environment as suggested by industry standards. Even in these instances, we noted 
the test environment was months behind the production environment. For a test 
environment to be effective, it should mirror the production environment. As of July 
2010, the test environment had not been updated since March 2010, which does not 
meet IT standards. Data changes were made to a number of properties between those 
months, so testing may not identify all anomalies prior to acceptance. 


The other half of the changes we reviewed were tested directly in the working version of 
the system. The issue with this is the new change could negatively impact the system, 
leaving Orion nonoperational. In fact, division staff notified us of an earlier incident 
where staff tested a change directly in the production environment and it crashed the 
system and took a week to completely recover. These type of issues could be prevented 
by implementing industry approved practices for change control, including request, 
management approval, system development, testing, and acceptance. 


Orion Experiences Technical Problems 


After identifying control weaknesses that could affect system and data integrity, 
we performed additional work to identify any potential impact. After reviewing 
documentation detailing the changes made to Orion since implementation, we found 
there have been over 620 technical system problems including calculation and data 
handling errors. Of the more than 620 technical problems identified, about 69 percent 
were rated as high or critical priority by the agency (See Figure 5). 


Figure 5 
Orion Technical Problem Prioritization 
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Source: Compiled by Legislative Audit Division from information obtained from the 
Department of Revenue. 





In addition, our review noted 21 problems relating specifically to calculating issues. In 
fact, during the annual assessment period for 2010, two additional calculation errors 
were identified including the unexpected rounding of .5 acres to whole acres. For 
example, if a property was 9.5 acres, the system was rounding the acreage to 10. ‘The 
department has recognized the issue and is in the process of fixing it. Another ongoing 
issue that is occurring during the current assessment period is erroneous Value Before 
Reappraisal (VBR) during the process to output condominium values. At this time, 
the department has not been able to resolve the problem and is manually calculating 
VBRs for residential condominium properties. 


Although we cannot point to the specific incidents that have caused each problem, 
we noted the issues are ongoing. The number of problems per month has gone down 
slightly but still remains fairly steady at about 10 per month. Department staff are also 
aware of the ongoing possibility of miscalculation, which forces them to manually 
review thousands of assessment notices for irregularities before they are mailed out. 
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RECOMMENDATION #6 





We recommend the Department of Revenue: 


A. | Implement and follow a complete change control process that meets 
industry standards. 


B. Ensure the test environment reflects the current Orion production 
environment. 


To 


Uncontrolled Access Presents Risk 
to Integrity of Orion Data 


Users of Orion must request access to the system through the Information Technology 
and Processing Division. At the time of the audit, there were over 400 users of the 
Orion database throughout the state. During our review of user access to Orion, we 
found the department has controls in place limiting access through management 
approval. However, we identified two areas where uncontrolled user access presents 
a risk to Orion data integrity. The first area of uncontrolled access we identified is 
the assignment of “user configurable” access to department personnel. The vendor 
programmed in this access allowing users the ability to modify fields, processes, and 
core system calculations. Although there are instances where this access is necessary, 
it is equivalent to making changes to the internal coding of the system and should 
be controlled through both limited access and change control. We did not identify 
any controls limiting this access according to business need or how the access should 
be used. Additionally, system changes made using “user configurable” access do not 
go through the normal change control process. The end result is users with ability 
to make unauthorized changes to Orion, which can result in processing errors or 
favorable results for the user. 


Another area of uncontrolled access we identified is the assignment of “override” 
capabilities to all field appraisers, managers and division staff. Override allows varying 
levels of access, but essentially allows the user to override property assessment data 
after the data has been entered and processed. After reviewing the access list, at least 
325 out of 426 users have some level of override access. When an appraisal is appealed 
and the original appraisal is overturned, a new value has to be added to the record. 
Since the new value is not based on the internal calculations of the system, it cannot 
be recalculated, so a manual override has to be used to update the final assessment. As 
a result, the original calculated value is overwritten and not maintained in the system. 


There are fields in Orion that let the user denote if the override is associated with an 
appeal. Audit work identified approximately 53,000 overrides of property assessment 
record information occurred since Orion was implemented. Further work noted 
many of these were justified because they were associated with an appeal. However, 
we identified over 2,400 overrides only defined as “other” and “miscellaneous.” Based 
on Orion information, we could not verify the appropriateness of these overrides. 
Department management is also not retroactively reviewing overrides for correctness 
and authorization. The high number of users with this access and lack of documentation 
of reasons for overrides present a risk to data integrity. This access is not controlled and 
could cause data errors or be used to create favorable results for the user. 
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RECOMMENDATION #7 





We recommend the Department of Revenue: 


A. Limit system user access for “user configurability” and “override” 
capabilities. 


B. Require all system changes to go through established change control 
procedures. 


Oo 


Develop and implement management procedures to review instances of 
“overrides” for correctness and authorization. 


TL 


Orion Integrity Would be Strengthened by Input Controls 


One feature of the Orion system is it allows for input edits to be set for data entry 
fields. Input edits are system controls that will not allow data to be saved if critical 
elements are missing or not in the correct format. During the audit, we noted the 
majority of available field edits have not been activated. Through review of Orion data, 
we identified 105 property records with missing data required to calculate assessments. 
We also identified over 62,000 current residential properties with missing addresses 
and over 6,100 property records with inspection dates that are in the future or precede 
1980. Finally, we identified 25 property records with negative final values, which were 
identified to us as the result of data entry omissions and errors. These data errors could 
have been avoided with effective input edits. 


According to the department, the primary reason for not triggering the edits is because 
there are limited system resources and the edits will use even more. While the resource 
issue is a legitimate concern, as discussed in the next section, standards still expect 
controls to be in place to ensure data validity, which includes completeness, accuracy, 
and consistency. In addition, the implementation of input edits will help improve the 
documentation inconsistencies discussed in Chapter III. 


CO QS 


25 


26 | Montana Legislative Audit Division Legislative Audit Division 


RECOMMENDATION #8 





We recommend the Department of Revenue activate input controls to ensure 
complete and accurate entry of data into Orion. 


To 


Improved Hardware Capabilities will 
Increase Orion Performance 


As mentioned in the previous section, Orion users expressed concern with system 
resources. Limited system resources results in downtime and slow processing due to 
system stoppages. IAAO standards suggest hardware should be powerful enough to 
permit computerization of appropriate application. During a two-month period of our 
audit, there were 19 instances where Orion had technical difficulties, such as slow or 
frozen systems. We noted staff workload was affected by Orion technical problems 
including lost data. As discussed in Chapter II, Orion data does not always match 
hardcopy reappraisal data located in field offices, so these instances could impact the 
accuracy of property records. 


After speaking with agency staff and reviewing the IT architecture, we noted all major 
components of Orion, including the test, production, and reporting databases, are 
stored on a single server, which is not able to adequately handle all usage. As a result, 
Orion experiences downtime and other technical issues. 


RECOMMENDATION #9 





We recommend the Department of Revenue review hardware capabilities to 
determine if Orion can more appropriately handle processing. 
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RECEIVED 


Tori Hunthausen 


Legislative Auditor, Legislative Audit Divisi 
-caliative ee ee ive Audit Division SEP 13 2010 
P.O. Box 201705 LEGISLATIVE AUDIT DIV. 


Helena, MT 59620-1705 
Dear Ms. Hunthausen: 


Thank you for the opportunity to respond to the recommendations in the Property Reappraisal Process 
Performance Audit of the Montana Department of Revenue. Our response to the recommendations raised in 
the report is as follows: 


Recommendation #1 
We recommend the Department of Revenue strengthen controls for documenting reappraisal activities. 


Concur. The department agrees that there is always room for improvement with regard to documenting 
decisions. To help strengthen controls for documentation of reappraisal activities, the department is: 
e developing an administrative procedure to bolster the documentation of valuation approach 
selection, 
e starting to utilize mobile, field tablet computers to improve appraiser efficiency and documentation by 
completing the transition from paper to electronic property records, and 
e improving Orion computer system controls (see responses to recommendations 6, 7, 8 and 9). 


Recommendation #2 
We recommend the Department of Revenue define expectations for each minimum number for property 
inspections to be conducted during each reappraisal cycle. 


Concur. The department agrees that it can better focus staff attention on physical inspections by clarifying 
expectations for this critical work. However, varying rates of parcel growth relative to staff during an 
appraisal cycle will result in some variance in the level of physical inspections. Historically, the primary 
obstacle to inspecting all properties is staffing levels in the face of large growth in the number of parcels 
across Montana. The department will continue to work diligently to conduct regular physical inspections of 
property as consistently as possible across the state. 


Recommendation #3 
We recommend the Department of Revenue comply with current procedure for department reviews. 


Concur. The department will rewrite its current policy to reflect the realities of staff levels in relation to 
workload and the number of informal reviews and appeals. The department will implement steps to improve 
communication with taxpayers on the timing of review processes. The department will also take steps to 
attain the level of review performance required in the revised policies. 
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Recommendation #4 
We recommend the Department of Revenue establish standardized methods for documenting supervisory 
review. 


Concur. The department will improve and standardize the current process of documentation. These 
improvements will begin with centralizing and standardizing the several existing electronic and paper 
processes that are used in the current supervisory review process. The department is also investigating 
computer software options to formalize this documentation process. 


Recommendation #5 
We recommend the Department of Revenue identify, compile, and analyze management information to 
improve the property reappraisal process. 


Concur. The department has set as a priority the development of analytical and management information 
systems to improve the guidance of all its functions, including property assessment. The department 
recognizes the limitations of the current Orion computer system and is in the process of utilizing other tools 
to increase data management. 


Recommendation #6 
We recommend the Department of Revenue: 
A. Implement and follow a complete change control process that meets industry standards. 


B. Ensure the test environment reflects the current Orion production environment. 


Concur. The department is currently implementing a formalized change control process that complies with 
the accepted system life cycle development practices, with adherence to the Software Engineering Institute’s 
Capability Maturity Model (SEI CMM) guidelines. In addition, the department recognizes the need for a 
viable test environment and has created a test environment that closely resembles our production hardware 
configuration but not a current version of the data. In addition, the department is actively pursuing an 
automated overlay process that will copy production data to that test environment. 


Recommendation #7 
We recommend the Department of Revenue: 
A. Limit system user access for “user configurability” and “override” capabilities. 


B. Require all system changes to go through established change control procedures. 


C. Develop and implement management procedures to review instances of ‘overrides’ for correctness 
and authorization. 


Concur. The Orion application was developed to be user configurable; however, this option does not 
conform to industry change control standards. The department is currently pursuing a course of action that 
will limit user configurable options and follow an approved change control process. 


There are multiple levels of override capabilities in Orion. Based on business needs, various security 
controls can be implemented in Orion that support a statewide business plan for override methods and 
standards. Additionally, reports can be developed to support a formal review and control process. The 
department is actively pursuing these controls. 
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Recommendation #8 
We recommend the Department of Revenue activate input controls to ensure complete and accurate entry of 
data into Orion. 


Concur. The department recognizes a need for in-system data entry constraints in Orion and is currently 
working on implementing a solution. Since the Orion application is commercial off-the-shelf software, most 
data entry edits are configured on a site by site basis. In addition, stronger data entry rules should improve 
efficiency of department staff and reduce costs while increasing the confidence in the quality of data that 
exists in Orion. 


Recommendation #9 
We recommend the Department of Revenue review hardware capabilities to determine if Orion can more 
appropriately handle processing. 


Concur. Montana is one of two states performing property reappraisal at a statewide level. Montana has 
had to implement numerous customizations in order to run the Orion commercial off-the-shelf application 
statewide. The department is committed to continuous improvements to the Orion application and 
undertaking several tasks to accomplish this objective. First, the department is initiating a third party 
architectural review of the Orion system. Second, the reporting database is being migrated to a separate 
database server from the operational database. Third, an automated job scheduling functionality based on 
job dependency is being developed for the Orion application. Finally, the department is requesting that the 
vendor for Orion, Tyler Technologies, modify Orion to work on sixty-four bit architecture. The current thirty- 
two bit configuration limits our ability to appropriately scale the hardware. 


Sincé 


Dan Gee 


Director 
PO Box 5805 
Helena, MT 59604-5805 





